OSS-TMM: Guidelines for Improving the Testing Process of Open Source Software

Open Source Software (OSS) products do not usually follow the traditional software engineering development paradigms found in textbooks. Specifically, testing activities in OSS development may be quite different from those carried out in Closed Source Software (CSS) development, also due to the fact that OSS processes often seem to be less structured than CSS ones. Since testing and, in general, verification may require a good deal of resources in OSS, it is necessary to have ways for assessing and improving OSS testing processes. In this paper, we provide a set of testing guidelines and testing issues that OSS developers may want to take into account to decide which testing techniques make most sense for their OSS products. This paper 1) provides a checklist that helps OSS developers identify the most useful testing techniques according to the main characteristics of their products under test and 2) outlines a proposal for a method that helps assess the maturity of OSS testing processes. The method is a proposal of a Maturity Model for testing processes (which we call OSS-TMM). To show its usefulness, we have applied the method to seven real-life projects. Specifically, we present in the paper how we applied the method to BusyBox, Apache Httpd and Eclipse Test & Performance Tools Platform to show how the checklist supports and guides the testing process of these OSS products.